Memory use discussion

Dear all,

Doors 8.0 user, Win XP SP2, 4 Gb RAM
I am facing memory problem when executing DXL script of our own (modules/objects transformation preparing document generation). Script is complex enough and module objects wide enough to I now am unable to execute to the end without dxl error.
Errors are:

  • Memory exhausted dxl error
  • incorrect regular expression (Out of memory)
where given line is: if(matches("not trivial regexp"...

Questions are:
  • Is there Memory leak ?
  • Could it be a wide use of string variables ?
(do not use Buffer but Skips and Arrays)
  • Could it be a wide use of regular expression ?
And is 'Regexp' better than 'matches()' ?

  • Does someone experienced Doors on Windows XP 64 bits ?

Thanks for your help.

Kind regards.
Vincent
vimo - Fri Sep 04 14:57:29 EDT 2009

Re: Memory use discussion
Strad - Mon Sep 07 12:10:27 EDT 2009

strings are probably the thing. The String table in DOORS doesn't empty so a DXL ninja will use strings as sparingly as possible...

It's also worth a quick note that DOORS afaik doesn't yet support large memory sizes, so even on a 64bit system with huge amounts of ram DOORS will cap out it's memory use at 2G. (IIRC there is a command line switch to enable DOORS to use 3Gig of RAM, but that's the limit)

Re: Memory use discussion
Peter_Albert - Tue Sep 08 04:31:06 EDT 2009

Strad - Mon Sep 07 12:10:27 EDT 2009
strings are probably the thing. The String table in DOORS doesn't empty so a DXL ninja will use strings as sparingly as possible...

It's also worth a quick note that DOORS afaik doesn't yet support large memory sizes, so even on a 64bit system with huge amounts of ram DOORS will cap out it's memory use at 2G. (IIRC there is a command line switch to enable DOORS to use 3Gig of RAM, but that's the limit)

Strad is right as regards the string table, but IIRC also the excessive definition of regular expressions within subroutines creates memory leaks, and, similarly important, it slows down the execution of the script significantly. If you have an issue on either memory or runtime, I'd suggest defining global regular expressions instead of defining them locally.

Peter

Re: Memory use discussion
dpechacek - Tue Sep 08 08:59:47 EDT 2009

Peter_Albert - Tue Sep 08 04:31:06 EDT 2009
Strad is right as regards the string table, but IIRC also the excessive definition of regular expressions within subroutines creates memory leaks, and, similarly important, it slows down the execution of the script significantly. If you have an issue on either memory or runtime, I'd suggest defining global regular expressions instead of defining them locally.

Peter

Not to mention if you open modules which have layout DXL or which are linked to modules which get opened in the background which do, that DXL also executes and clutters the string table. I have seen DOORS not release memory even on a module with no DXL. It is quite frustrating. I wish instead of making DOORS prettier or adding features few use, they'd focus on cleaning up memory and get to work enabling garbage collection in DXL. Of course with the Jazz platform now, they'll probably just ignore it since when DOORS gets fully implemented in Jazz, DXL will no longer be based on the same codebase as it is now.

AAI Services, Textron
dpechacek@aai.textron.com
David.Pechacek@gmail.com

Re: Memory use discussion
vimo - Wed Sep 09 07:20:37 EDT 2009

dpechacek - Tue Sep 08 08:59:47 EDT 2009
Not to mention if you open modules which have layout DXL or which are linked to modules which get opened in the background which do, that DXL also executes and clutters the string table. I have seen DOORS not release memory even on a module with no DXL. It is quite frustrating. I wish instead of making DOORS prettier or adding features few use, they'd focus on cleaning up memory and get to work enabling garbage collection in DXL. Of course with the Jazz platform now, they'll probably just ignore it since when DOORS gets fully implemented in Jazz, DXL will no longer be based on the same codebase as it is now.

AAI Services, Textron
dpechacek@aai.textron.com
David.Pechacek@gmail.com

I thank you all. I will use your advices to improve this script.

br, Vincent

Re: Memory use discussion
llandale - Thu Sep 10 12:05:34 EDT 2009

Albert and Pechacek are right.

Its either too much string manipulation or too many modules open at once, or both.

Static regexp should be defined globally since using the 'regexp' command is VERY string intensive and VERY slow. Dynamic but frequently used regexp can be defined once and stored in a global skip using the string as the key, then search the skip before defining the regexp next time.

Use Buffers frequently. However, creating and deleting a Buffer is VERY slow; frequently called functions that stage in Buffers can define and create one 'globally' that only that function uses, saving a lot of time.

Buffer g_bufGetValue = create()
string GetValue(Object obj)
{  g_bufGetValue = ""     // Erase Buffer
   .. use g_bufGetValue
   return(stringOf(g_bufGetValue))
}  // end GetValue()

 


Understand that when you open a module, you also open its link modules and often other modules, invisibly, as well. You should fairly frequently close out all residually open modules in your big-scoped scripts.

 

 

 

  • Louie

 

 

Re: Memory use discussion
vimo - Thu Sep 10 12:42:35 EDT 2009

llandale - Thu Sep 10 12:05:34 EDT 2009

Albert and Pechacek are right.

Its either too much string manipulation or too many modules open at once, or both.

Static regexp should be defined globally since using the 'regexp' command is VERY string intensive and VERY slow. Dynamic but frequently used regexp can be defined once and stored in a global skip using the string as the key, then search the skip before defining the regexp next time.

Use Buffers frequently. However, creating and deleting a Buffer is VERY slow; frequently called functions that stage in Buffers can define and create one 'globally' that only that function uses, saving a lot of time.

Buffer g_bufGetValue = create()
string GetValue(Object obj)
{  g_bufGetValue = ""     // Erase Buffer
   .. use g_bufGetValue
   return(stringOf(g_bufGetValue))
}  // end GetValue()

 


Understand that when you open a module, you also open its link modules and often other modules, invisibly, as well. You should fairly frequently close out all residually open modules in your big-scoped scripts.

 

 

 

  • Louie

 

 

Thanks Louie,

You confirm that I should work on regexp implementation too. This script often use matches() inside frequent loops and I should easily gain to write them using Rexexp outside the loops.

thanks, nice to read you all, Vincent